Wristwatch based interface for augmented reality eyewear

ABSTRACT

Augmented reality eyewear devices allow users to experience a version of our “real” physical world augmented with virtual objects. Augmented reality eyewear may present a user with a graphical user interface that appears to be in the airspace directly in front of the user thereby encouraging the user to interact with virtual objects in socially undesirable ways, such as by making sweeping hand gestures in the airspace in front of the user. Anchoring various input mechanisms or the graphical user interface of an augmented reality eyewear application to a wristwatch may allow a user to interact with an augmented reality eyewear device in a more socially acceptable manner. Combining the displays of a smartwatch and an augmented reality eyewear device into a single graphical user interface may provide enhanced display function and more responsive gestural input.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/743,410, filed on Jan. 15, 2020 which is a continuation of U.S.patent application Ser. No. 15/686,512, filed on Aug. 25, 2017, each ofwhich are incorporated herein by reference in their entireties.

BACKGROUND OF THE INVENTION

Augmented reality is a version of our “real” physical world augmentedwith virtual images. Augmented reality may be experienced using variousdevices such as smartphones, tablets, or augmented reality eyewear.Augmented reality eyewear allows a user to experience augmented realitythat may encompass much or all of a user's field of view, withoutrequiring a user to hold a device such as a smartphone or tablet, thusfreeing the user's hands for interaction with virtual or real objects. Auser wearing augmented reality eyewear may see virtual images displayedin their field of view. Some virtual images may move relative to thereal world as a user move's their head. Other images may appear to befixed in position relative to the real world.

A user of augmented reality eyewear may interact with augmented realityby making hand gestures. For example, a user may swipe vertically inorder to scroll a web browser. However, current methods of gesturalinput for augmented reality eyewear may be socially awkward or may notbe intuitive. For example, a third party generally will not be able tosee the virtual images that are observed by a user wearing augmentedreality eyewear. Such a third party may be confused or disturbed whenobserving an augmented reality eyewear user swiping, poking, or pinchingat the air in front of the user. There is thus a need for a sociallyacceptable and intuitive method of gestural input for augmented realityeyewear.

BRIEF SUMMARY OF THE INVENTION

Described herein are systems and methods for providing a wristwatchbased interface for augmented reality eyewear.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of an augmented reality eyewear devicesensing the positioning of a wristwatch worn by a user.

FIG. 2 is a flowchart of a sequence for calibrating augmented realityeyewear to a wristwatch.

FIG. 3 illustrates an example of an application selection interfaceanchored to a user's wristwatch.

FIG. 3 a is a flowchart of a sequence for anchoring the graphical userinterface of an application selection interface to a user's wristwatch.

FIG. 4 illustrates an example of a music player application interfaceanchored to a user's wristwatch.

FIG. 5 illustrates an example of a web browser application interfaceanchored to a user's wristwatch.

FIG. 6 illustrates an example of a dot lock interface anchored to auser's wristwatch.

FIG. 7 illustrates a user simulating twisting a crown of a wristwatch inorder to interact with a web browser application.

FIG. 7 a is a flowchart of a sequence for recognizing and implementingtraditional wristwatch gestures.

FIG. 8 illustrates a user simulating rotating a bezel of a wristwatch inorder to interact with an application selection interface.

FIG. 9 illustrates a menu interface, as such would appear to a userwearing augmented reality eyewear.

FIG. 10 illustrates a wristwatch being used as a simulated planar shiftinput interface.

FIG. 10 a is a flowchart of a sequence for interpreting planar shiftgestures made in the vicinity of a wristwatch.

FIG. 11 illustrates a user interacting with a scaled duplicateinterface.

FIG. 11 a is a flowchart for creating and interacting with a scaledduplicate interface.

FIG. 12 illustrates an example of a combined smartwatch and augmentedreality eyewear display, as such would appear to a user wearingaugmented reality eyewear.

FIG. 13 illustrates an example of a smartwatch and augmented realityeyewear display, combined into a continuous display.

FIG. 13 a is a flowchart for combining the displays of an augmentedreality eyewear device and a smartwatch into a continuous display.

FIG. 14 is a flowchart for a sequence to choose between input from anaugmented reality eyewear device and a smartwatch.

FIG. 15 illustrates an example of a 3D launcher interface.

DETAILED DESCRIPTION OF THE INVENTION

Various embodiments and aspects of the disclosure will be described withreferences to details discussed below, and the accompanying drawingswill illustrate the various embodiments. The following description anddrawings are illustrative of the disclosure and are not to be construedas limiting the disclosure. Numerous specific details are described toprovide a thorough understanding of various embodiments of the presentdisclosure. However, in certain instances, well-known or conventionaldetails are not described in order to provide a concise discussion ofembodiments of the present disclosure.

Some portions of the detailed descriptions which follow are presented interms of algorithms, sequences, or processes which include operations ondata stored within a computer memory. The operations typically requireor involve physical manipulations of physical quantities. Usually,though not necessarily, these quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated.

Unless specifically stated otherwise as apparent from the followingdiscussion, it is appreciated that throughout the description,discussions utilizing terms such as “processing” or “computing” or“recognizing” or “calculating” or “determining” or “displaying” or“receiving or the like can refer to the action and process of a dataprocessing system, or similar electronic device, that manipulates andtransforms data represented as physical (electronic) quantities withinthe system's registers and memories into other data similarlyrepresented as physical quantities within the system's memories orregisters or other such information storage, transmission or displaydevices.

The present disclosure can relate to an apparatus for performing one ormore of the operation described herein. This apparatus may be speciallyconstructed for the required purpose, or it may comprise a generalpurpose computer selectively activated or reconfigured by a computerprogram stored in the computer. Such a computer program may be stored ina machine-readable storage medium.

Augmented Reality Eyewear

Augmented reality eyewear may create an augmented reality space byoverlaying virtual images onto the physical world. The augmentation ofreality may also be subtractive, and remove from view objects that existin the physical world. The augmentation of reality may modify theappearance of physical objects in the world or create entirely newvirtual objects or phenomenon. The field of view of an augmented realityeyewear device may be referred to as the scene, which may also includenon-visual phenomenon that may be detected by various sensors of anaugmented reality eyewear device.

Augmented reality eyewear may display images in various ways. Augmentedreality eyewear may display virtual images on a transparent orsemi-transparent surface that rests in front of the user's field ofview. Alternatively, display of virtual images may be accomplished byprojecting light into the eye of the user. As augmented realitytechnology advances, various other methods of displaying virtual objectto a user may be utilized.

Augmented reality eyewear may include sensing components, processingcomponents, display components, power source components, interfacecomponents, speaker components, and data transmission components.

Sensing components may allow augmented reality eyewear to sense objectsand phenomenon in the physical world. For example, augmented realityeyewear may contain visual or quasi-visual sensors such as visible lightcameras, time-of-flight cameras, infrared sensors, structured lightsensors, laser sensors, micro-radar sensors, or sonar sensors. Somesensors may be passive sensors, such as visible light or infraredsensors, in that they do not emit. Other sensors may be active sensors,such as laser or micro-radar, in that they both emit and receive.Augmented reality eyewear may contain sensors for sensing non-visualdata. For example, augmented reality eyewear may contain audio sensors,sonar sensors, an inertial measurement unit, accelerometers, gyroscopes,GPS, magnetometers, altimeters, heartrate sensors, temperature sensors,and various biometric sensors. Sensing components may allow augmentedreality eyewear to sense data regarding the positioning and movement ofthe user and the user's eyes.

Data collected by sensing components may be passed to processingcomponents for processing. Processing components may include a centralprocessing unit, a graphics processing unit, active memory and memorystorage. Processing components may dictate the functioning of othercomponents.

Display components may display virtual images to the user of theaugmented reality eyewear. Display components may project a virtualimage onto a transparent or semi-transparent lens in front of the user'seye such that the user sees the virtual image as well as the realphysical world behind said image. Alternatively, display components mayproject virtual images into the eye of the user, again allowing the userto see the virtual image and the real physical world. Alternatively,display components may produce a holographic image.

An augmented reality eyewear device may also include power sourcecomponents, such as a battery or solar panels, that provide power to thedevice.

An augmented reality eyewear device may also include interfacecomponents that allow the user to interface with the device. Forexample, various buttons, touch surfaces, sliders, or microphones thatmay be positioned on the device.

An augmented reality eyewear device may also include data transmittingcomponents. Data transmitting components may transmit and receive data,thereby exchanging data with other devices or the internet. Suchtransmitters and receivers may include cellular radio, Bluetooth, NearField Communication, and WiFi.

An augmented reality eyewear device may also include audio components,such as traditional speakers, bone conduction speakers, hypersonicspeakers or directional speakers for presenting audio information to theuser.

Tracking and Recognition

Augmented reality eyewear may utilize various sensors and methods knownin the art to track and recognize objects in the real world. For atleast a decade, augmented reality devices have utilized marker basedtracking to track and recognize objects. More recently, augmentedreality devices have utilized markerless tracking. As augmented realityeyewear devices evolve, new methods for tracking and recognizing two andthree dimensional objects and gestures may emerge. It is appreciatedthat various sensors and processing techniques may be utilized in theembodiments described herein to recognize and tracking objects andgestures.

It is also appreciated that augmented reality eyewear may recognize andtrack a wristwatch by recognizing and tracking other objects in theenvironment. For example, an augmented reality eyewear device may inferthe location of a user's wristwatch if the augmented reality eyeweardevice knows the location of the user's hand and the location of theuser's sleeve.

FIG. 1 illustrates an example of an augmented reality eyewear devicesensing the positioning of a wristwatch worn by a user. In FIG. 1 a userwears an augmented reality eyewear device 101 and a wristwatch 102. Atime-of-flight sensing component of the augmented reality eyewear deviceand a visible-light sensing camera sense visible light radiation 103reflected off of the wristwatch. Processing components of the augmentedreality eyewear device process the data collected by the sensingcomponents in order to model the shape of the wristwatch and theposition of the wristwatch in 3-dimensional space.

Calibrating the Augmented Reality Eyewear

The augmented reality eyewear device may be calibrated in order to moreefficiently recognize a specific wristwatch. One method of calibrationmay be to create a 3D model of a specific wristwatch and to derive many2D images from such a 3D model. Multiview stereo reconstruction may beone method of producing a 3D model of a wristwatch worn by a user.Modeling of a wristwatch may occur passively or augmented realityeyewear may prompt a user to present the watch in a certain manner androtate the watch. For example, augmented reality eyewear may prompt theuser to place the watch-bearing wrist in the center of the field of viewof the augmented reality eyewear and also prompt the user to point tothe watch with the non-watch-bearing hand. Some methods of modeling suchas feature point modeling may provide efficiency advantages. Once awristwatch is modeled, augmented reality eyewear may be calibrated suchthat it may more easily recognize a specific wristwatch and may morequickly and accurately determine the position and orientation of awristwatch.

Alternatively, a 3D model of a wristwatch, or other types of calibrationdata, may be provided to an augmented reality eyewear device. Forexample, a smartwatch may transmit a 3D model of itself to augmentedreality eyewear, allowing for faster calibration than if the augmentedreality eyewear device had to model the smartwatch from scratch.Augmented reality eyewear may also search a user's purchase history onan online database to determine whether a user has purchased a watch. Ifa user has purchased a watch and the watch worn by the user matches awatch in the purchase history, augmented reality eyewear may downloadmodeling and calibration data for a purchased watch.

FIG. 2 is a flowchart of a sequence 200 for calibrating augmentedreality eyewear to a wristwatch. In step 201 a user wears an augmentedreality eyewear device and a wristwatch and a calibration sequence isinitiated. In step 202 the augmented reality eyewear device may receiveinput from peripheral devices or devices connected wirelessly.

In step 203 the augmented reality eyewear device enters smartwatchdetection mode. If a smartwatch is connected via Bluetooth or WiFi Awareor NFC or any other wireless protocol and the smartwatch supports aprotocol that allows the sharing of visual tracking calibration data,then the augmented reality eyewear device obtains calibration data fromthe smartwatch and the sequence is completed 204 if a smartwatch isconnected via Bluetooth or WiFi Aware or NEC or any other wirelessprotocol, but the smartwatch does not support a protocol that allows thesharing of visual tracking calibration data, then the augmented realityeyewear device attempts to obtain data from the smartwatch that wouldallow it to determine the make and model of the smartwatch. If theaugmented reality eyewear device is successful in determining the makeand model of the smartwatch it attempts to obtain calibration data usingonline images of said smartwatch or to download calibration data from anonline database. If the augmented reality eyewear device obtainscalibration data in this manner the sequence is completed.

While in Smartwatch Detection Mode, if the augmented reality eyeweardevice is not already connected to a smartwatch, it performs a scan ofBluetooth, NFC, WiFi Aware or other wireless protocols to see if asmartwatch is nearby. If multiple wireless signals are detected nearbythe augmented reality eyewear device performs the following subsequence,beginning with the signal with the highest signal strength and endingwith the signal with the lowest signal strength: If the signal strengthis below a certain threshold, indicating that there is no reasonablechance that the smartwatch is currently being worn on the wrist of auser, then the augmented reality eyewear device moves on to the nextsignal. If the signal strength is not below the threshold, the augmentedreality eyewear examines the signal and uses a database to determinewhether the signal comes from a smartwatch and attempts to determine themake and model of the smartwatch. If the signal does not belong to asmartwatch, the augmented reality eyewear moves on to the next signal.If the signal does belong to a smartwatch, the augmented reality eyewearprompts the user, asking whether the smartwatch that has been detectedis the user's smartwatch. If the smartwatch does not match the user'ssmartwatch, the augmented reality eyewear moves to the next signal. Ifthe smartwatch does match the user's smartwatch, the augmented realityeyewear device may prompt the user further in order to determine makeand model if make and model data were not determined earlier. Once makeand model are determined, the smartwatch will attempt to obtaincalibration data in any manner previously discussed and if calibrationdata is successfully obtained that sequence will be completed.

If the augmented reality eyewear is unable to obtain calibration data inSmartwatch Detection Mode, step 203, it enters Purchase History Mode,step 205. If the augmented reality eyewear device has access to theuser's purchase history for an online retailer or proprietary ecosystemit performs the following subsequence while in Purchase History Mode:The device sets a time limit. The device, in a “parallel for loop,”searches for images of a watch in the user's purchase history andcompares such images to images of the scene, which may include a watch.This process continues until the predetermined time limit is reached, atwhich point the device moves to step 207. If a watch in the purchasehistory is matched to a watch in the device's current environment beforethe time limit is reached, the augmented reality eyewear device attemptsto obtain calibration data in any manner previously discussed. Ifcalibration data is obtained the sequence is completed 206. For optimalperformance, it is recommended that step 205 and step 207 shall beperformed simultaneously, on separate threads.

In step 207 the augmented reality eyewear device enters Automatic ModeOn-Wrist. In this step, the augmented reality eyewear device attempts tovisually model a user's wristwatch without active input from the user.The device first sets a time limit such as 30 second. If it is knownthat the user always wears their watch on their left wrist, theaugmented reality eyewear device will search the scene for the user'sleft hand. If it is known that the user always wears their watch ontheir right wrist, the augmented reality eyewear device will search thescene for the user's right hand. If the user is known to wear a watch onboth wrists, or their preference is not known, the augmented realityeyewear device will search for either hand simultaneously. In searchingfor a hand, the augmented reality eyewear will set allowable retries foreach hand to a certain number. If the frame rate of the computer visionsystem is slow, such as 1 frame per second, then an appropriate numberof allowable retries might be in the approximate realm of 5 to 10retries. If the frame rate of the computer vision system is fast, suchas 60 frames per second, then an appropriate number of allowable retriesmight be in the approximate realm of 300 to 600 retries. When a hand isdetected the augmented reality eyewear device may search an area nearthe hand for anything that looks like a wrist or a watch. If a watch isidentified, the augmented reality eyewear device will attempt toconstruct a 3D model of the watch, update the watch calibration data,and complete the sequence 208. If no watch is detected, the value forallowable retries will be reduced by 1 for the hand near which a watchwas searched for. Once allowable retries for both the left and righthand are reduced to zero or once the time limit has elapsed, theaugmented reality eyewear device moves to step 209.

In step 209 the augmented reality eyewear device enters Interactive ModeOn-Wrist. The device first asks the user if the user is wearing a watch.If the user answers in the negative, the device sets a variableindicating that the user is not wearing a watch and the sequence iscompleted 210. If the user answers in the affirmative, the deviceprompts the user to present or point out the watch while it is beingworn. For example, one way to do this would be to ask the user to usetheir non-watch-bearing hand to point at the watch. The devicesimultaneously searches for cues such as; the image of a watch, a handmaking a pointing gesture, or a hand not making a gesture. If the devicerecognizes a hand making a pointing gesture or a hand not making apointing gesture the device searches near such cues for a wrist orwatch. If a wrist is recognized, the device looks more intensely for awatch. If a watch is not located, the device continues searching for atime limit. If no watch is found in the time limit, the device moves tostep 211. If a watch is located during the Interactive Mode On-Wrist,the device sets an allowable number of re-prompts and begins 3Dreconstruction. The device prompts the user to rotate the user'swatch-bearing wrist as needed to reconstruct the entire watch. If thereconstruction is successful, the device's calibration data is updatedand the sequence 210 is completed. If the reconstruction is notsuccessful, the device reduces the number of allowable re-prompts andprompts the user again. Once the number of allowable re-prompts reacheszero, the device moves to step 211.

In step 211 the device enters Interactive Mode Off-Wrist. In this mode,the device prompts the user to remove the user's watch and place thewatch on a flat surface such as a table or counter. The device mayindicate to the user that for best results, the surfaced should be asolid color, not patterned or textured. The device may then prompt theuser to begin taking multiple photographs of the watch either usingtheir augmented reality eyewear device or using their smartphone,whatever they are more comfortable with. The augmented reality eyeweardevice may prompt the user to take the first photograph from a vantagepoint directly over the center of the watch face, looking down at it. Ifa watch is not detected, the augmented reality eyewear device willprompt the user to attempt the first photograph again. If a watch isdetected by the augmented reality eyewear device, the augmented realityeyewear device begins 3D reconstruction, starting with the firstphotograph. 3D reconstruction continues as the augmented reality eyewearprompts the user to take photos of the watch from all angles. Even as auser is prompted to take photographs, the augmented reality eyeweardevice may take photographs continually, even when the user is notmanually taking photographs, in order to collect a lot more calibrationdata with less user effort. If any specific vantage points have missingor low-quality information the augmented reality eyewear device mayprompt the user to take a photograph from that specific angle. Theinstructions may be verbal or graphical or both. Graphical instructionsmay be pictographic drawings that were created by a human in advance, orthey may consist of 3D AR imagery that is generated on the fly and maybe shown on the glasses or the smartphone. Once enough images aregathered to create a sufficiently accurate 3D reconstruction, theaugmented reality eyewear device alerts the user that calibration iscomplete, updates its calibration data and completes sequence 212. IfInteractive Mode Off-Wrist is unsuccessful, sequence 213 is completedwithout updating calibration data.

Wristwatch Anchored Graphical User Interface

In some current applications of augmented reality eyewear, virtualimages displayed by the augmented reality eyewear device are fixed withrespect to the device and thus move in relation to the real physicalworld as the user moves or turns their head. However, virtual images mayalso be anchored to real objects in the physical world, such that thevirtual image maintains its positioning in relation to a real physicalobject even as the user and the real physical object move. An example ofaugmented reality anchoring would be a virtual image of a baseball cap,overlaid onto the head of a human. As the human moves or turns, thevirtual baseball cap would move in a corresponding fashion such that theillusion of the baseball cap being worn by the human is maintained asthe human moves. Maintaining said illusion may require the virtual imageto move in three-dimensional space, rotate, as well as adjusting size,color, shading and clarity,

FIG. 3 illustrates an example of a graphical user interface anchored toa user's wristwatch, as such would appear to a user wearing augmentedreality eyewear. In FIG. 3 a user wearing an augmented reality eyeweardevice is looking at the face of a wristwatch 102. The augmented realityeyewear application may have a graphical user interface. Such agraphical user interface may be anchored to a real physical object suchas a wristwatch worn by the user. One example of a graphical userinterface anchored to a user's wristwatch may be an applicationselection interface. The augmented reality eyewear may display a seriesof application icons 301 in the airspace near the forearm and hand ofthe user. As displaying icons for all available applications may beimpractical, the application interface may show a subset of availableapplication icons and indicate that more application icons areavailable. The user may interact with the virtual icons in a similarmanner as a user would interact with virtual icons on the touchscreendisplay of a smartphone or tablet. For example, the user may select anapplication to run by tapping at the location in three-dimensional spacewhere the virtual application icon appears. The user may also swipe leftor right to view more application icons.

When a user of augmented reality eyewear focuses his gaze on hiswristwatch, third parties may intuitively understand that the user isinteracting with a device. If, on the other hand, the application iconswere simply displayed in the empty airspace directly in front of theuser, the user would be required to swipe or tap in the empty airspacein front of the user. In such a situation, a third party may mistakenlybelieve that the user is trying to interact with said third party oranother person or object that exists in the real world. By anchoring thegraphical user interface to a wristwatch, a wearer of augmented realityeyewear may reduce the incidence of awkward or confusing interactionswith third parties. Moreover, many users and third parties may befamiliar and comfortable with individuals interacting with smartwatches.When a user makes a swiping or tapping motion in the vicinity of awristwatch, the gestures may not appear to be particularly unusual.

FIG. 3 a is a flowchart of a sequence 302 for anchoring the graphicaluser interface of an application selection interface to a user'swristwatch. In step 303 a user wears an augmented reality eyewear deviceand a wristwatch and activates an application selection interface. Instep 304 the augmented reality eyewear device determines the positionand orientation of the wristwatch. The augmented reality eyewear devicemay utilize calibration data or data provided by a smartwatches inertialmeasurement unit to assist in determining the position and orientationof the wristwatch. In step 305 the augmented reality eyewear devicerenders a graphical user interface. In step 306 the augmented realityeyewear device displays an application interface such that applicationicons appear in a horizontal line across the forearm of the user withone application icon above the watchface. The augmented reality eyeweardevice continuously determines 307 the position and orientation of thewristwatch and adjusts 308 the image of the graphical user interfaceaccordingly. The augmented reality eyewear device repeats 309determining 307 the position and orientation of the watch and adjusting308 the image accordingly until an application is selected or theapplication selection interface is terminated.

FIG. 4 illustrates an example of a music player application interfaceanchored to a user's wristwatch 102, as such would appear to a userwearing augmented reality eyewear. In FIG. 4 , a graphical userinterface of a music player application is displayed by an augmentedreality eyewear device. Control buttons for the music playerapplication, such as track-back 401, play-pause 402, and track forward403 may be anchored to the user's wristwatch and displayed in theairspace near the user's forearm, the user's wristwatch and the user'shand. Information regarding the track 404, album 405, and artist 406 maybe displayed above the wristwatch. The user may interact with thevirtual controls by making a button-pressing gesture in proximity to thevirtual button.

FIG. 5 illustrates an example of a web browser application interfaceanchored to a user's wristwatch 102, as such would appear to a userwearing augmented reality eyewear. In FIG. 5 , a graphical userinterface 501 of a web browser application is displayed by the augmentedreality eyewear device. The web browser window may be displayed abovethe user's wristwatch 102 in a plane that is substantially perpendicularto the watch face. The user may again interact with the web browser in asimilar fashion as a user might interact with a web browser on thetouchscreen of a smartphone or tablet with multi-touch functions. A usermay interact with the web browser application by tapping, pinching tozoom, scrolling 502, or swiping. A user may also use three-dimensionalgestures to interact with the graphical user interface. For example, inorder to close a window, a user could make a crumpling and tossinggesture to simulate crumpling and tossing away a piece of paper.

FIG. 6 illustrates an example of a dot lock interface 601 anchored to auser's wristwatch 102, as such would appear to a user wearing augmentedreality eyewear. In FIG. 6 the graphical user interface of a dot lockapplication is displayed in the airspace in front of the user'swristwatch 102. Dot lock screens are used to prevent unauthorized accessto electronic devices with touchscreens. To unlock such a device, a userdraws a pattern between a number of dots. Here, the user may draw such apattern between virtual dots in the airspace in front of the user'swristwatch. The dot lock virtual image may also be three dimensional,creating a more secure dot lock password.

The anchoring of the GUI to the wristwatch may be made more realistic byreducing the latency from when the watch and/or eyewear moves to whenthe rendered graphics are redrawn. Several techniques may be used tosubstantially reduce this latency:

-   a. In the eyewear's one or more cameras used for tracking, the    exposure time may be lowered. Although this would make a photo    viewed by a human look bad, for a computer vision algorithm, that is    not a concern,-   b. In the eyewear's one or more cameras used for tracking, the    resolution may be reduced to the minimum amount acceptable by the    computer vision algorithm. The resolution may be adaptively    increased again later, temporarily, if the vision algorithm loses    confidence in its tracking, then reduced back to the lower    resolution after the vision algorithm regains confidence.-   c. In the eyewear's one or more cameras used for tracking, the    “cropping” or “windowing” setting may be utilized to only look at a    small region surrounding the watch. Looking at fewer pixels' worth    of area allows those pixels to be read more times per second within    a given image-processing pipeline of fixed bandwidth.-   d. When writing to the eyewear's one or more displays, the eyewear    may write only the pixel rows and columns that are strictly    necessary to display the GUI, without wasting any time writing to    rows or columns that are all black,-   e. When writing to the eyewear's one or more displays the eyewear    may utilize interlacing. That is, the eyewear may write only to    odd-numbered rows on one frame/field, then only to even-numbered    rows on the next frame/field, over and over, allowing twice as many    frames/fields to be displayed per second.-   f. The eyewear's overall tracking and rendering system may utilize    the technique known in the art as “racing the beam”. This means that    while the eyewear scans a particular row with the one or more    cameras, it may immediately update the one or more rows on the one    or more displays that correspond positionally to that camera row.    This avoids any unnecessary latency that would be incurred if a less    optimal algorithm was used that updated rows in the one or more    displays corresponding to a position that had been scanned by the    one or more cameras many milliseconds ago.-   g. If the watch being tracked is a smartwatch and contains its own    IMU, the eyewear may utilize the latest IMU readings coming from the    smartwatch to update the rendered position of the graphics    “blindly”. That is, if the eyewear possesses IMU readings that are    more recent than the latest camera-based visual tracking data, then    the eyewear may take it on faith that the IMU readings are accurate,    and update the displayed graphics in accordance with the IMU    updates. If it later turns out that the IMU readings were    inaccurate, then the displayed graphics may be corrected at that    later time, possibly resulting in a small “jump” as the graphics    jump from the incorrect position to the correct position.

Traditional Wristwatch Gestures

Users of traditional wristwatches may interact with a wristwatch in anumber of ways. A user may turn a crown to adjust the time, press abutton to start or stop a timer, or rotate a bezel. Such movements mayfeel intuitive or natural for a user operating a graphical interfacethat is anchored to a wristwatch.

FIG. 7 illustrates a user simulating a crown-twisting motion in order tointeract with web browser application, as such would appear to a userwearing augmented reality eyewear. In FIG. 7 a graphical user interface501 of a web browser is displayed in the airspace above a user'swristwatch 102. The user may simulate twisting 701 the crown 702 of awristwatch in order to interact with the web browser. For example, bysimulating twisting 701 the crown 702 of the wristwatch 102, the usermay cause the graphical user interface 501 web browser to scroll 703.

FIG. 7 a is a flowchart of a sequence 704 for recognizing animplementing traditional wristwatch gestures. A user wears an augmentedreality eyewear device that is running an application and wears awristwatch. In step 705 the user initiates the sequence by making a handgesture with the non-watch-bearing hand. In step 706 the augmentedreality eyewear determines whether a gesture is made within a certaindistance of the watch. If a gesture is made in proximity of thewristwatch, the augmented reality eyewear moves to step 708. If agesture is not made in proximity of the wristwatch, the sequence iscompleted 707. In step 708 the augmented reality eyewear compares thegesture to a traditional-wristwatch-gesture-library. Thetraditional-wristwatch-gesture-library may contain models of gesturesthat a user of wristwatch may traditionally make, such as twisting acrown, pressing a button, or rotating a bezel. If the gesture matches amodel from the traditional-wristwatch-gesture-library, that model ispassed 709 to the application for processing and the sequence iscompleted 7010. If the gesture does not match a model from thetraditional-wristwatch-gesture-library, the sequence is completed 7011.

FIG. 8 illustrates a user rotating the bezel of a wristwatch to interactwith a web browser application, as such would appear to a user wearingaugmented reality eyewear. In FIG. 8 a graphical user interface of anapplication selection interface is displayed in the airspace above auser's wristwatch 102. The user may simulate rotating 801 the bezel 802of the wristwatch in order to interact with the web browser. Forexample, by simulating rotating 801 the bezel 802 of a wristwatch 102,the user may cause application icons 301 to scroll 803 horizontally.

Wrist Gestures

A user may interact with a wristwatch anchored graphical user interfacewith the hand that is not wearing the wristwatch. A user may alsointeract with an augmented reality eyewear device by making gestureswith the wrist that wears the wristwatch.

FIG. 9 illustrates a menu interface, as such would appear to a userwearing augmented reality eyewear. A user may access such a menuinterface by bringing the watch face of a wristwatch 102 into the centerof the user's field of view, as if the user were checking the time on astandard wristwatch. The augmented reality eyewear may sense thisgesture and may respond by activating the augmented reality display anddisplaying a the graphical user interface 901 of a menu interface to theuser. A menu interface may contain application icons 902, a powerfunction 903, a calibration function 904, a battery indicator 905 andtime and date information 906. Of course, a menu interface may containvarious combinations of applications, functions and system information.If the user removes the wristwatch from the field of view of theaugmented reality eyewear before making any selections within the menuinterface, the augmented reality display may be deactivated. On theother hand, depending on the user's input within the menu interface,augmented reality display functions may or may not continue after thewristwatch is removed from the field of view of the augmented realityeyewear. An augmented reality eyewear device may possess a wrist-gesturedatabase and perform a sequence similar to the sequence discussed abovefor traditional wristwatch gestures in order to recognize and interpretwrist-gestures.

Another example of a gesture made with the hand that wears thewristwatch would be a quick wrist-twisting gesture, mimicking a gesturethat might be made by a user to adjust the placement of the wristwatchon the wrist. Such a gesture may be used as a swift activation gesturefor an application such as a camera application that users may want toactivate immediately, without having to go through a menu or applicationselection interface.

Additionally, a user may make gestures that utilize both hands in orderto interact with real or virtual objects. For example, a user may be ina room that contains multiple smart appliances such as one or more smartlight bulbs and a smart television. A user may select between such smartappliances by first bringing the wristwatch into the scene such that theface of the wristwatch is pointing upwards. A user may then bring theuser's other hand into the scene and make a pointing gesture, by layingthe user's index finger across the face of the wristwatch such that theuser's index finger points in the direction of the desired smartappliance. If for example, the user points to a smart light bulb in thisfashion, the smart light bulb may turn on or off, or a menu may appearby which a user may dim the bulb or change its color. This method ofselection by pointing may be useful in various instances such as if auser is far away from a real object that the user would like to select.For example, a user may see a restaurant from across a street and desireto view reviews for the restaurant. By making a pointing gesture acrossthe face of the user's wristwatch, a user may select said restaurant andinteract with information related to the restaurant.

The augmented reality eyewear device may allow the user to programgestures to activate various applications or perform functions.Additionally, certain apps may be allowed to attempt to register arecommended gesture or set of gestures, which the user may give or denypermission to do.

Simulated Planar Shift Input Interface

Users of traditional desktop computers interface with a computer'sgraphical user interface using a mouse and keyboard. The mouse is anexample of a planar shift input interface. The graphical user interfaceof a traditional computer is presented in a vertical two-dimensionalplane via a display monitor. The mouse or touchpad operates in ahorizontal two-dimensional plane. Thus, when a user moves the mouseforward, the cursor on the graphical user interface moves up and whenthe user moves the mouse backward, the cursor on the graphical userinterface moves the cursor downward.

FIG. 10 illustrates a wristwatch being used as a simulated planar shiftinput interface. In FIG. 10 a user wears an augmented reality eyeweardevice 101 and a standard wristwatch 102 on the user's left wrist. Theuser may position the wristwatch 102 in front of the user with the watchface pointing upwards. In FIG. 10 the augmented reality eyewear device101 displays a graphical user interface 501 of a web browser window inplane above the face of the user's wristwatch 102 in a plane that issubstantially perpendicular to the face of the wristwatch 102. The usermay interact with the web browser window by moving the user's right hand1001 in the horizontal plane above the user's wristwatch and below theweb browser. If the user is wearing a smartwatch, the touchscreen of thesmartwatch may be used in this manner similarly to a touchpad on alaptop computer.

FIG. 10 a is a flowchart of a sequence 1002 for interpreting planarshift gestures made in the vicinity of a wristwatch. A user wears anaugmented reality eyewear device and a wristwatch. In step 1003 anapplication is initialized. The augmented reality eyewear devicedetermines 1004 the position and orientation of the user's watch. Instep 1005 the augmented reality eyewear displays the graphical userinterface of the application in a plane perpendicular to the face of thewatch. In step 1006 the augmented reality eyewear device recognizes ahand gesture made in the parallel plane above the watch face. Theaugmented reality eyewear then translates 1007 the gesture from agesture that is parallel to the watch face to a gesture that is parallelto the web browser's graphical user interface. The application theninterprets 1008 the gesture, which is now parallel to the web browser'sgraphical user interface and the sequence is completed 1009. The planarshift need not be perpendicular or even substantially perpendicular. Forexample, while the input plane may be parallel to the watch face, theplane of the graphical user interface may be at an angle of 60 degreeswith respect to the watch face for a better viewing angle.

Scaled Duplicate Interface

In various augmented reality eyewear applications, a user may wish tointeract with a three-dimensional object. For example, a furnituredesigner may wish to rotate a virtual three-dimensional model of akitchen table in order to view the model from different angles. If themodel is enlarged such that it encompasses a large portion of a user'sfield of view, intuitive interaction with such a model may cause a userto make grand, sweeping gestures that are in proportion to the scale ofthe model. For example, an intuitive method of rotating a full-sized,virtual kitchen table may be to extend the right arm all the way to theright and swipe all the way across the body to the left. A scaledduplicate interface may allow a user to interact with a largethree-dimensional virtual object without making such sweeping gestures.

FIG. 11 illustrates a user interacting with a scaled duplicateinterface. In FIG. 11 , the user wears an augmented reality eyeweardevice 101 and a wristwatch 102. The augmented reality eyewear device101 may display a larger three-dimensional image 1101 in front of theuser and a scaled-down version 1102 of said three-dimensional image 1101above the watch face of a user's wristwatch 102. The user may theninteract with the scaled version 1102 of the image by making smallergestures in the three-dimensional space above the user's wristwatch 102.The scaled down version of the object may not be an exact duplicate ofthe object. It may be of lower resolution to conserve processing poweror be shown as a wire frame to indicate to the user that the scaled-downversion is to be used as a control. A menu interface may be displayednear the scaled-down version that allows the user to interact with thefull-sized object in various ways. For example, an option available in amenu may be a paint-wheel by which a user may change the color of anobject.

FIG. 11 a is a flowchart of a sequence 1103 for creating and interactingwith a scaled duplicate interface. A user wears an augmented realityeyewear device and a wristwatch. In step 1104 a scaled duplicateinterface is initiated. The augmented reality eyewear determines 1105the position and orientation of the user's watch. In step 1106 theaugmented reality eyewear renders a virtual object. In step 1107 theaugmented reality eyewear displays a full-sized version of the object inthe airspace in front of the user. In step 1108 the augmented realityeyewear displays a scaled-down version of the object above the face ofthe user's watch. In step 1109 the augmented reality eyewear recognizesa gesture made in proximity to the scaled down object. The applicationinterprets the gesture as though it were made in a location and scalerelative to the full-sized image and adjusts 1110 both the full-sizedobject and scaled-down object accordingly. The sequence is thencompleted 1111. This sequence may be continually repeated for the entiretime that a scaled duplicate interface is desired by a user.

A scaled duplicate interface may be activated and associated with aparticular object in a number of ways. As a first example, when a useradds a new virtual object to the scene, the scaled duplicate interfacemay be activated and may be linked to the new virtual object. As asecond example, if a user were walking through the physical world andcame into close proximity with a object that the user could control orinfluence, the scaled duplicate interface may be activated and may belinked with the object that is now in close proximity. In this secondexample, the object that comes into proximity may be either a virtual ora real object. As a third example, a user may actively select aparticular virtual or real object. A user may actively select an objectby voice command, through a menu interface, through an applicationlauncher interface, or by making a gesture, such as pointing, inrelation to the object. Non-gestural means of selection, such as voicecommand or selection from a menu interface may be of particular use inselecting objects that are not within the scene, such as a scaledduplicate version of a user's home while a user on vacation.

In the instances where the scaled duplicate image is an image of a realphysical object, the scaled duplicate interface may automatically updateits appearance to match the appearance of the real object. For example,if the object is the user's self-driving automobile, the scaledduplicate image may be adjusted based on the movements of the physicalcar. When the car goes around a curve, the wheels on thescaled-duplicate image may move in corresponding fashion and theorientation of the car may shift. Attempts by the user to control theinterface may or may not control the real object. For example, a usermay open the trunk of the vehicle by tapping on the trunk of thescaled-duplicate image. A scaled duplicate interface may be particularlyuseful in industrial applications. For example, robotic arms couldmanipulate a real physical object in corresponding fashion to a user'smanipulation of a scaled-duplicate image of said real physical object.

Augmented Reality Eyewear Interaction With Smartwatches

A user need not wear a specially designed wristwatch in order tointeract with augmented reality eyewear through a wristwatch anchoreduser interface. A user may have various different styles of watchdepending on aesthetic preference and may change watches as frequentlyas desired. In fact, a user may wear any analog or digital wristwatchand experience many of the benefits of a wristwatch-anchored interface.As virtual images and interfaces are overlaid onto a user's standardwristwatch, the wristwatch may act as a simulated smartwatch and theuser may interact with their standard wristwatch as though it were asmartwatch.

A user may also wear a “real” smartwatch to interact with an anchoredinterface of an augmented reality eyewear device. A smartwatch's owncomponents may contribute to the utility of the augmented realityeyewear device. For example, a smartwatch's inertial measurement unitmay communicate via Bluetooth with the augmented reality eyewear devicein order to assist the augmented reality eyewear device in determiningthe location and orientation of the smartwatch. Moreover, augmentedreality eyewear devices may be slimmer, less expensive, and theirbatteries may last longer if the eyewear relies on a smartwatch forsensing or processing.

A smartwatch may contain a heartrate monitor that is capable of trackinga user's heartrate. Such heartrate monitors ray usephotoplethysmography, pressure sensors, or other sensing components tomeasure changes in blood flow and blood pressure near the user's wrist.Heartrate data may then be passed to an augmented reality eyewear devicethat is worn by the user and the augmented reality eyewear device maydisplay a real-time heartrate monitor along with various statistics thatmay be derived from the user's heartrate. The displayed information maybe displayed either in close proximity to the watch, or in a fixed partof the user's view through the augmented reality eyewear. For example,heartrate information may be displayed in the upper-left corner of thescene. A smartwatch may also be equipped with sensors to monitor pulseoximetry, blood glucose data, body temperature, galvanic skin responseor other biometric data. Sensors may even be able to detect the onset ofseizures or of diseases such as colds or fevers. Such data may bedisplayed by augmented reality eyewear in similar fashion as heartratedata. Additionally, important alerts, such as a diabetic having lowblood glucose, may pop up near the center of the user's vision to getthe user's attention. Augmented reality eyewear software and displaysshould be designed such that a user's field of vision is not undulyobstructed while the user is engaged in dangerous activities such asdriving or operating machinery.

Combined Smartwatch and Augmented Reality Eyewear Displays

Current augmented reality eyewear technology may not be capable ofproducing images with as much brightness, color contrast, and resolutionas current smartwatch displays. Therefore, a user that wears both asmartwatch and an augmented reality eyewear device may prefer to use thesmartwatch display for certain functions. Similarly, the touchscreensensors of a smartwatch may more accurately capture finger movement thanthe sensing components of an augmented reality eyewear device.

FIG. 12 illustrates an example of a combined smartwatch and augmentedreality eyewear display, as such would appear to a user wearingaugmented reality eyewear. In FIG. 12 , a user wears an augmentedreality eyewear device and a smartwatch and is engaged in playing amilitary strategy game. The augmented reality eyewear device may displaya map 1202 of a battlefield in the airspace above the user's smartwatch.In FIG. 12 , various icons 1203 representing military personal andvehicles are displayed on the battlefield map. When a user selects on ofthese icons 1203, statistics and information relating to the selecteditem may appear on the user's smartwatch 1201. For example, in FIG. 12 ,the user has selected a Humvee icon 1204 on the battlefield andstatistics for the Humvee, including the Humvee's occupancy, fuel range,and armor rating are displayed on the user's smartwatch 1201. As such,the two devices may display complimentary elements of the graphical userinterface of a single application. A user may interact with thegraphical user interface both by touching the smartwatch and by makinggestures with respect to the augmented reality display.

The displays of a smartwatch and augmented reality may also be combinedas one continuous display. FIG. 13 illustrates an example of asmartwatch and augmented reality eyewear display, combined into acontinuous display. In FIG. 13 , a navigation application is runsimultaneously on a smartwatch 1201 and an augmented reality eyeweardevice. A portion of the graphical user interface is displayed by theaugmented reality eyewear device and a portion is displayed by thesmartwatch 1201 such that the displays appear as one continuous display.

For example, augmented reality eyewear may display the map portion of agraphical user interface 1301 of a rideshare application. As the usermoves the face of the smartwatch 1201 into the area in which thegraphical user interface 1301 of the map is displayed, the augmentedreality eyewear may cease to display the portion of the graphical userinterface that is occupied by the smartwatch and the smartwatch mayinstead display the corresponding portion of the graphical userinterface. The user may control both the augmented reality eyeweardisplay and the smartwatch display by, for example, swiping or pinchingthe display of the smartwatch or by making hand gestures in the air inclose proximity to either the smartwatch or to the graphics displayed bythe eyewear. Some smartwatch displays may be brighter and more vividthan some augmented reality eyewear displays. The edges of thesmartwatch display may be faded such that the transition from smartwatchdisplay to augmented reality eyewear display is more gradual or aperimeter of a predetermined width may be left blank around the outlineof the smartwatch. Alternatively, augmented reality eyewear may simplydim the portion of the display that would appear above the display ofthe smartwatch.

FIG. 13 a is a flowchart of a sequence 1302 for combining the displaysof an augmented reality eyewear device and a smartwatch into acontinuous display. A user wears an augmented reality eyewear device anda wristwatch. In step 1303 a combined-continuous-display sequence isinitiated. The augmented reality eyewear device and smartwatch establish1304 a connection. In step 1305 the augmented reality eyewear devicechecks to determine whether the desired application is running on bothdevices. If the application is not running on a device, that devicelaunches 1306 the application. Once both devices are running the desiredapplication, the eyewear determines 1307 the position and orientation ofthe smartwatch. In step 1308 the smartwatch renders a graphical userinterface to be displayed on the smartwatch. The smartwatch sends 1309the rendering to the eyewear. In step 1310 the eyewear renders agraphical user interface to be displayed on the eyewear and subtractsthe smartwatch rendering from the eyewear rendering. The smartwatch thendisplays 1311 the rendering of the smartwatch graphical user interfaceand the eyewear displays 1312 the rendering of the eyewear graphicaluser interface with the smartwatch graphical user interface beingsubtracted such that the displays of both devices appear as onecontinuous display. As the devices move in relation to each other, theeyewear will need to continuously determine the position of thesmartwatch and subtract appropriate portions of its own GUI in order tomaintain the appearance of one continuous display across both devices.The sequence is then completed. This sequence may be continuallyrepeated for the entire time that a combined display is desired.

Various configurations of application processing may be utilized by asmartwatch and augmented reality eyewear device in order tosimultaneously display the same application. As described above, bothdevices may run the full version of the application simultaneously.Alternatively, the devices may interact as a master-slave, in that oneperforms the processing and controls the display of another device.Similarly, one device may perform a majority, but not all of theapplication processing.

Combining the display of a smartwatch with the display of augmentedreality eyewear may present substantial benefits for applications suchas web browsers, map applications, or games with virtual terrain or avirtual map.

Tracking Gestures in Combined Displays

When two devices are used to simultaneously run the same application,both devices may detect input from a user such as gestures or touch. Inorder to continue to run the application, the devices must resolve thequestion of which input should control the application. Somesmartwatches may be able to detect gesture made in the vicinity of thesmartwatch, even if the watch is not touched, by using sensors similarto those used by augmented reality eyewear.

FIG. 14 is a flowchart of a sequence 1400 for reconciling between inputfrom an augmented reality eyewear device and a smartwatch. In step 1401an augmented reality eyewear device and a smartwatch simultaneously runand display an application. If the smartwatch registers 1402 a touch,the application selects 1403 the input from the smartwatch and nofurther reconciliation is needed and the sequence is completed 1404. Ifno touch is registered by the smartwatch and the augmented realityeyewear determines 1405 whether it has registered an air gesture. If theeyewear has registered an air gesture, the smartwatch determines 1407whether it has registered an air gesture. If the smartwatch has alsoregistered an air gesture each device interprets the gesture andascribes a confidence value to the gesture, indicating the probabilitythat the interpreted gesture is the gesture that the user actually made.In step 1408, the devices compare the confidence values. If theconfidence of the smartwatch is greater than the confidence of theaugmented reality eyewear, the application selects 1409 the input fromthe smartwatch and the sequence is completed 1410. If the confidence ofthe smartwatch is no greater than the confidence of the augmentedreality eyewear, the application selects 1411 the input from theaugmented reality eyewear and the sequence is completed 1412. If thesmartwatch does not register an air gesture, but the augmented realityeyewear does register an air gesture, the application selects 1413 theinput from the augmented reality eyewear and the sequence is completed1414. If no air gesture is registered by the augmented reality eyewear,the devices move to step 1417. In step 1417 the smartwatch determineswhether it has registered an air gesture. If the smartwatch hasregistered an air gesture the application selects 1418 the smartwatchinput and the sequence is completed 1419. If the smartwatch determinesthat it has not registered an air gesture, the sequence is completed1420.

3D Application Launcher

Devices capable of running multiple applications and functions may havea launching interface by which a user may view data output and mayselect applications or functions for the device to run. A launchinginterface of an augmented reality eyewear device may be displayed inthree-dimensional space. A launching function may display informationsuch as time, battery life and Internet connectivity. A launchinginterface may also display application icons that a user may tap orswipe to launch an application. A launching interface may also displayrecently viewed or bookmarked webpages.

A launching interface may also display widgets, whose appearance anddata output may change. An example of a widget may be a weather widgetthat appears as a three-dimensional cloud when the weather is cloudy, athree dimensional sun when the weather is sunny or a three dimensionallightning bolt when a thunder storm is in progress. A user may interactwith a widget without leaving the launcher interface. For example,augmented reality eyewear may display a 3D widget of a user'sautomobile. A user may be able to start the car from the launcherinterface by making a key-turning gesture near the car widget, or openthe trunk of the car by tapping the trunk on the car widget. In anotherexample, a user may be able to turn on the user's sprinkler system bytapping a sprinkler widget. In turn, the sprinkler widget may displaywater droplets to indicate that the sprinkler system was successfullyturned on. Given that a launcher may contain many icons and widgets,widgets may appear too small for a user to interact with in a granularway, such as tapping the trunk of a car widget. A user may enlarge aparticular widget for easier interaction by tapping once on the widget.

When a user wears an augmented reality eyewear device along with awristwatch, the launching interface of the augmented reality eyeweardevice may relate to the user's wristwatch in various ways. In oneexample icons, widgets, and device status data may appear to float asthree-dimensional objects in proximity to a user's wristwatch. Suchobjects may maintain a fixed position and orientation relative to thewrist upon which the watch is worn. Alternatively, such objects maymaintain a fixed position and orientation relative to the user'swristwatch. Alternatively, such objects may maintain a fixed positionrelative to the wristwatch bearing wrist, but the orientation of suchobjects may be fixed relative to another reference point such as thehorizon or the user's head. Alternatively, objects may be positioned andorientated based on a combination of the above-mentioned methods. Forexample, when a user rotates their wrist, an object may rotate, but therotation may be to a lesser degree in order for the object to be moreclosely oriented with another point of reference such as the horizon orthe user's head.

In FIG. 15 an augmented reality eyewear device displays a 3D launcherinterface in proximity to a user's wristwatch. In FIG. 15 , the 3Dlauncher interface displays a car widget 1501, a camera icon 1502 and amail icon 1503. In FIG. 15 , the headlights of the car widget 1501appear to be in an on position, demonstrating that the headlights of theuser's corresponding car in the real world are on.

Smarthome

Various smart devices within homes, offices and other buildings maypossess the capability of communicating with the internet or otherdevices. For example, on a hot summer day, a thermostat may communicatewith automated windows such that windows are opened in the evening, onceoutside temperatures drop, in order to cool the house without using airconditioning. An augmented reality eyewear device may be configured tocommunicate with and monitor such smart devices. An augmented realityeyewear device may display an interactive map of a user's home or officethat identifies various smart devices. A user may tap a smart device onsaid may to view more detailed information regarding the device andadjust device settings. When a user taps a smart device information,icons and settings may appear to float above the map, if the map is twodimensional, or outside of the map, if the map is three dimensional.

In various applications that involve maps, such as strategy games andnavigation applications, users may interact with points on a map in asimilar fashion as described above for a smart home application.Selecting an item or location on a map may cause objects and informationto pop up and appear to float above or near the item that was selectedon the map.

CONCLUSIONS RAMIFICATIONS AND SCOPE

Accordingly, the reader will see that anchoring the graphical userinterface of an augmented reality eyewear device to a wristwatch orusing a wristwatch as a means of input for an augmented reality eyeweardevice may improve the functioning of augmented reality eyewear andallow for more socially acceptable and more intuitive gestural input.The reader will also see that combining the displays of a smartwatch andan augmented reality eyewear device presents various benefits.Furthermore, advantages of a wristwatch based interface for an augmentedreality eyewear device may include:

-   -   a. An intuitive means of user input and interaction with        augmented reality eyewear.    -   b. A means of input that does not require the user to make        gestures in open airspace directly in front of the user, which        may lead to awkward social interactions and confusion. The means        of input described herein may simulate more socially acceptable        gestures such as a user adjusting a standard wristwatch or        interacting with a smartwatch.    -   c. A planar shift input mechanism using gestural input for        augmented reality eyewear. Such an input mechanism may be        intuitive for users who have experience with desktop or laptop        computers.    -   d. A virtual control panel for various augmented reality        applications that is anchored to an item of fashion that many        users may already have, thus requiring no additional purchases        on behalf of the user. This also allows the user to dress in the        style and fashion of the user's choosing.    -   e. A method of interacting with large virtual objects by making        small gestures near a wristwatch instead of making sweeping        gestures in the open airspace in front of the user.

While the above description contains many specificities, these shouldnot be construed as limitations on the scope, but rather asexemplification of one or several embodiments thereof. Many othervariations and operations are possible. For example;

-   -   a. The interface of an augmented reality eyewear device may be        anchored to a bracelet, a sleeve cuff, a tattoo, or the user's        bare wrist or forearm. The augmented reality eyewear device may        even display a virtual image of a smartwatch on the user's bare        wrist that the user may interact with.    -   b. The interface of an augmented reality eyewear device may be        anchored to the palm of a user's hand. This would simulate a        user interacting with a smartphone or tablet. If the user has a        smartphone or tablet, the interface could be anchored to such a        device as well. This again would reduce awkward social        interactions and the confusion of third parties.    -   c. Various components of the augmented reality eyewear device        may be decentralized or may no longer be necessary as technology        advances. For example, an augmented reality eyewear device may        rely on the processing power of a user's smartphone or on the        sensing components of a user's smartphone. Augmented reality        eyewear may have fewer or additional components than those        discussed herein. Moreover, components may be combined or        relocated.    -   d. Detachable sensing or emitting components may be affixed to a        wristwatch in order to increase the speed and accuracy at which        an augmented reality eyewear device models the location of the        wristwatch.    -   e. The disclosure sets forth various examples of computer        applications that may benefit from the disclosed systems and        methods. However, many other computer applications may benefit        from the disclosed systems and methods.

Accordingly, the scope should be determined not by the embodimentsillustrated, but by the appended claims and their legal equivalents.

What is claimed is:
 1. An augmented reality eyewear device implementedmethod for user interface comprising: detecting a selection of a targetobject, the target object being a full-scale virtual representation of aphysical object; responsive to detecting selection of the target object:determining positional information of a wrist of an end-user, thepositional information comprising orientation of the wrist of theend-user; displaying a scaled down virtual representation of the targetobject in proximity to the wrist of the end-user; recognizing one ormore gestures made by the end-user in proximity to the scaled downvirtual representation of the target object; and interpreting the one ormore end-user gestures as though they were made in a location and ascale relative to the target object by modifying a presentation of thefull-scale virtual presentation of the physical object.
 2. The method ofclaim 1, wherein the positional information comprises location of wrist.3. The method of claim 1, further comprising: adjusting the location andorientation of the target object according to the one or moreinterpreted gestures.
 4. The method of claim 1, further comprising:displaying a menu interface in proximity to the scaled down virtualrepresentation of the target object, wherein the menu interface providesone or more menu options for interacting with the target object;recognizing one or more end-user gestures in proximity to the menuinterface; interpreting the one or more end-user gestures in proximityto the menu interface as a selection of one or more of the menu options;and adjusting the target object in accordance with the one or moreselected menu options.
 5. The method of claim 1, wherein the scaled downvirtual representation of the target object is a scaled down wireframerepresentation of the target object.
 6. The method of claim 1, whereinthe scaled down virtual representation of the target object is a scaleddown, lower resolution representation of the target object.
 7. Themethod of claim 1 wherein the target object is a physical, internetconnected device.
 8. An eyewear device for presenting a user interfacewhile augmenting reality, the eyewear device comprising: one or moreprocessors; and a non-transitory computer readable storage mediumcomprising instructions that when executed by the one or processorscause the one or more processors to perform operations comprising:detecting a selection of a target object, the target object being avirtual representation of a physical object; determining positionalinformation relative to the target object, the positional informationcomprising orientation of a wrist of an end-user; displaying a firstvirtual representation of the target object in proximity the wrist ofthe end-user, the first virtual representation of the target objectbeing a scaled down virtual representation of the target object;displaying a second virtual representation of the target object inairspace in front of the eyewear device, the second virtualrepresentation of the target object being a full-scale virtualrepresentation of the target object; recognizing one or more gesturesmade by an end-user in proximity to the first virtual representation ofthe target object; and interpreting the one or more end-user gestures asthough they were made in a location and a scale relative to the secondvirtual representation of the target object.
 9. The system eyeweardevice of claim 8, wherein the positional information comprises locationof the physical anchor object wrist of the end-user.
 10. The eyeweardevice of claim 8, wherein the operations caused by instructionsexecuted by the one or processors further include adjusting the locationand orientation of the target object according to the one or moreinterpreted gestures.
 11. The eyewear device of claim 8, furthercomprising: displaying a menu interface in proximity to the scaled downvirtual representation of the target object, wherein the menu interfaceprovides one or more menu options for interacting with the targetobject; recognizing one or more user gestures in proximity to the menuinterface; interpreting the one or more end-user gestures in proximityto the menu interface as a selection of one or more of the menu options;and adjusting the target object in accordance with the one or moreselected menu options.
 12. The eyewear device of claim 8, wherein thescaled down virtual representation of the target object is a scaled downwireframe representation of the target object.
 13. The eyewear device ofclaim 8, wherein the scaled down virtual representation of the targetobject is a scaled down, lower resolution representation of the targetobject.
 14. A machine-readable non-transitory storage medium havinginstruction data executable by a machine to cause the machine to performoperations comprising: detecting a selection of a target object, thetarget object being a full-scale virtual representation of a physicalobject; responsive to detecting selection of the target object:determining positional information of a wrist of an end-user, thepositional information comprising orientation of the wrist of theend-user; displaying a scaled down virtual representation of the targetobject in proximity to the wrist of the user; recognizing one or moregestures made by the end-user in proximity to the scaled down virtualrepresentation of the target object; and interpreting the one or moreend-user gestures as though they were made in a location and a scalerelative to the target object by modifying the presentation of thefull-scale virtual presentation of the physical object.